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EXECUTIVE SUMMARY 


W 7 he U.S. economy will emerge from the Great 
Recession radically transformed from what 
it was a generation ago. Changes are afoot 
affecting which occupations and industry 
sectors will produce employment growth, 
as well as what education credentials, 
competencies, and skills will be required to do those 
jobs. These enduring shifts in labor market demand 
will compel laid-off workers to seek retraining for new 
careers, incumbent employees to upgrade their skills 
to remain competitive, and today's youth to reach 
for higher levels of education attainment to ensure a 
foothold in the middle class. 

The rising demand for education and training provides 
opportunities and also poses challenges for the 
nation's community colleges. If colleges expect to 
attract more students and graduate them prepared for 
sustainable careers, they must better align program 
offerings and course curricula to the needs of their 
local labor markets. Decisions on which programs to 
offer and what to teach in class should correspond 
to employers' immediate and ongoing demand for 
workers in a particular occupation or with a specific 
set of skills. 

Community colleges already take steps to address the 
workforce needs of local employers, but their efforts 
often are hampered by a lack of detailed, up-to-date 
information about occupations and skills in demand. 

A promising, yet still-evolving solution to that problem 
can be found within the large pool of job openings 
posted on the Internet. New sophisticated "spidering" 
and artificial intelligence technologies can aggregate 


and analyze these online job ads and provide a more 
comprehensive, "real-time" source of information 
about the hiring and skill needs of local employers. If 
proven accurate and reliable, analyses of online job ads 
could complement traditional ways that community 
colleges determine labor market demand for program 
and course offerings. 

A few colleges and states are leading the way in the 
use of online job ads to gather real-time intelligence 
about occupation and skill demands. For example, 
Northeastern states are building a methodology and 
dataset, based on online job ads, to determine demand 
for green jobs and skills. New Jersey community 
colleges are using online jobs as a first step for 
identifying opportunities to train jobless workers for 
difficult-to-fill jobs. Meanwhile, Florida's workforce 
development system is using this intelligence to adjust 
their lists of education and training programs that 
qualify for subsidies. Their experiences, documented 
in this report, demonstrate the possibilities and pitfalls 
with online job ads as a source of real-time labor 
market intelligence. 


NEW SKILLS FOR A NEW ECONOMY 


■ ' “ he United States is undergoing a significant transformation from the industrial economy of the mid- 

I | twentieth century to a service- and knowledge-based economy that depends on workers with advanced 
education and skills. On the one hand, the current recession has accelerated change, eliminating hundreds 
of thousands of low-skill, blue-collar jobs and swelling the unemployment rolls with workers who have only 
a high school education or less. On the other hand, economists with the Georgetown University Center 
on Education and the Workforce suggest that five occupational clusters will drive employment growth 
and demand for more highly skilled workers as the nation emerges from the economic downturn: managerial 
and professional office; education; health care professional and technical; scientific, technical, engineering, 
mathematics, and social sciences (STEM); and community services and arts (Carnevale, Smith, & Strohl 2010). 

The Georgetown economists predict that by 2018 nearly two-thirds of all available jobs will reguire at least some 
postsecondary education (Carnevale, Smith, & Strohl 2010). That's up from 3 in 10 jobs in 2008 as counted by 
the U.S. Bureau of Labor Statistics (Lacey & Wright 2010). Job growth is expected to touch occupations reguiring 
postsecondary education: an estimated 30 percent of new jobs will go to workers with some college or a two-year 
degree (Carnevale, Smith, & Strohl 2010). The creation of jobs reguiring an Associate's degree will outpace any 
other education and training category (Lacey & Wright 2010). 

A 2009 survey of the Business Roundtable substantiates the demand for college-educated workers: 65 percent 
of employers surveyed said that they require an Associate's degree or higher for most positions. Employers cited 
the greatest need for workers with more technical skills than ever before, higher degrees or certifications, and 
improved skills or better qualifications (Business Roundtable 2010). 

In recognition of the transforming economy and employers' workforce needs, the Obama Administration, 
along with several states, the Bill & Melinda Gates Foundation, Lumina Foundation for Education, and other 
private funders, have set ambitious goals for dramatically increasing the number of Americans who earn 
postsecondary credentials over the next 10 to 15 years (Obama 2010; Bill & Melinda Gates Foundation 2010). 

In addition, President Obama has called on all Americans to complete at least one year of postsecondary 
education or training (Obama 2009). The President and private funders have backed up their proclamations 
with new investments in postsecondary education, with the goal of expanding access and success, particularly in 
community colleges, long considered a low-cost gateway to a career or an advanced degree. 

The need for higher skills has not been lost on Americans. In the wake of the economy's downturn, enrollment is 
surging at community colleges: droves of unemployed workers have returned to school seeking retraining, while 
cash-strapped families are taking advantage of affordable tuition rates (Selingo 2009). Even before the economy 
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crashed, short-term, noncredit training programs had become popular. They now serve more students than 
traditional for-credit academic programs on many community colleges campuses (Van Noy et al. 2008). 


Mismatch Between School and Work 

Yet in the midst of rising ambitions and surging enrollments, there are growing concerns about whether 
postsecondary education institutions prepare students for skills and careers that are in demand. The media 
has been awash with personal stories of laid-off workers who have enrolled at community colleges and other 
education and training institutions only to remain unemployed upon graduation because their career choice or 
newly acguired skills did not meet the needs of employers (Goodman 2010). The promise of green jobs prodded 
many community colleges to develop training programs; however, in some areas, few jobs have materialized for 
graduates (Fletcher 2010). Meanwhile, employers have insisted that they are still struggling to find workers with 
the right skills (Whitehouse 2010; Motoko 2010). According to the Business Roundtable survey, more than 60 
percent of employers reported in July 2009 that they were having difficulty finding qualified applicants to fill 
current vacancies, even though millions of Americans had lost their jobs in the recession and were searching for 
work (Business Roundtable 2010). 

The mismatch between education offerings and employment opportunities has brought stiff competition 
for community colleges from proprietary and online training providers. Moreover, in some regions, four-year 
universities are now serving industries and occupations once dominated by Associate's degree holders: health 
care employers increasingly consider a Bachelor's degree as the minimum academic qualification for nurses, in 
particular for those without prior work experience (Jacobs 2011). In the meantime, states are tightening resources 
and raising accountability standards on community colleges and other postsecondary institutions (State Higher 
Education Executive Officers 2011; HCM Strategists n.d.). 

The bottom line: community colleges are under pressure to produce graduates who can get jobs. But identifying 
which skills and occupations are in demand is easier said than done. 


Difficulties in Identifying Employer Needs 

Taking steps to understand business needs is nothing new for community colleges. They commonly review 
government statistics about employment conditions and long-term occupational projections, as well as 
government databases tracking the knowledge, skills, and abilities needed to perform hundreds of occupations. 
Many colleges convene industry advisory groups in an effort to verify skill requirements and get assistance in 
developing curricula for professional and technical programs. Community colleges also participate in regional 
sector initiatives, partnering with local businesses, workforce and economic development agencies, and other 
community organizations to address the skill needs of critical industries. Colleges regularly tie curricula to 
workplace needs by delivering classroom instruction as part of apprenticeship programs, and they collaborate 
with individual employers to develop customized training solutions for incumbent workers or new hires. 

These strategies can uncover quite a bit about workforce needs, but they have limitations. In general, it is difficult 
to use government data sets to ascertain current and ongoing demand for specific occupations at the state or 
local level. Data sets tend to have sample sizes that are too small and dispersed to provide detailed analysis by 
industry, occupation, or geography. Data are collected too infrequently and released too slowly to offer timely 
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evidence of labor market changes; in some cases, data may be collected monthly but published quarterly and 
yearly. Government data sets also present challenges for reflecting the impact of market changes, such as 
emerging occupations and skills, along with job gains and losses, and they leave out some important information 
altogether, such as the attainment of certificates (Reamer 2010). 

Among the particular concerns: 

The taxonomies in the Occupational Information Network (0*NET) are updated every seven years, making it 
difficult to identify emerging skill requirements for occupations in a timely manner (Reamer 2010). 

The Job Openings and Labor Turnover Survey (JOLTS) and Business Employment Dynamics (BED) lack 
geographic detail on hiring trends (Reamer 2010). 

Long-term occupational projections are prone to error and underestimate the demand for postsecondary 
education (Carnevale, Smith, & Strohl 2010). Moreover, they provide too little detail to accurately reveal local 
trends (Reamer (2010). 3 

Meanwhile, industry advisory groups and regional sector strategies can be difficult and time-consuming to 
organize and maintain. With employers actively engaged, these groups can provide valuable insight about 
workforce needs, but they don't necessarily reflect broader trends within the local labor market; even the best 
advisory boards represent just a fraction of the total employers in a region or industry. Finally, apprenticeship 
programs can address employer needs for certain occupations, but they do not reach into certain industries or 
deep into particular local labor markets. 

Above and beyond these constraints, community colleges typically lack the resources to conduct the deep, 
rigorous analyses needed to match curricula with the skill needs of local employers. What is sorely missing is a 
dependable source of information that can efficiently and effectively diagnose not only real-time demand for 
occupations and skill requirements but also emerging trends throughout a local labor market. 
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"REALTIME” 


INSIGHT ON JOBS AND SKILLS 


D iscovering which local employers are hiring and what qualifications are needed for available jobs would 
be a significant step toward enabling community colleges to align program offerings with labor market 
demands. But until recently, it was impossible to gather information about these job vacancies in a 
timely, systematic manner. The Internet now provides an additional means for capturing hiring activity. 
Increasingly, employers are advertising job openings on their own websites or through online classifieds 
sites, government-run job banks, or fee-for-service job boards. Since 2006, employers have invested 
more in online outreach than in newspaper ads to recruit workers (Borrell Associates 2009). Recent research 
suggests that jobs advertised online now reflect at least 70 percent of all openings. 

Recognizing the treasure trove of data being produced, enterprising for-profit companies have developed 
sophisticated technologies that can aggregate and analyze these online job ads and provide "real-time" 
intelligence on hiring trends for industries and occupations, job requirements (e.g., skills, education, experience, 
certifications), and compensation (e.g., salary and benefits). 

Analyses of online job ads offer several advantages over traditional sources of data and information on hiring 
trends: 

Aggregating millions of online job ads can provide broad, detailed, and timely information. 

Because much of the process is automated, job ads can be collected and analyzed quickly for a fraction of the 
cost of other approaches. 

A great deal of information can be extracted from job ads, including in many cases specialized skills and 
certifications. 

With this knowledge, community colleges can access information on local hiring trends and skill reguirements. 
They can keep up with the changing demands of the labor market by incorporating new or emerging skills and 
certifications into existing curricula while designing and delivering new programs of study. In addition, analyses 
of online job ads can inform the design of career pathway curricula by identifying common and related skill 
requirements in job openings representing broad occupational clusters (Goldberger 2010). College counselors 
can use the information to guide students toward program and course offerings tied to occupations with strong 
employment and earning potential. 
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Three Steps for Analyzing Online Job Ads 

Several steps are involved in gathering actionable, real-time data from online job advertisements: 

Collect job advertisements posted on a wide range of online boards. 

De-duplicate those ads to ensure an accurate picture of available job openings by occupation and 

industry. 

Step Extract key information from job descriptions and analyze it to gain insight on job requirements and 
trends. 

The quality and range of services available to analyze the real-time market are undergoing constant change, 
as new entrants, tools, and systems are introduced. Table 1 lists firms in this nascent field and the services they 
provide. As of June 2011, Career Builder had just released a new tool, the Supply and Demand Portal, while Simply 
Hired was enhancing its services. 


TABLE 1. 

CAPABILITIES AND ANALYTIC PROWESS OF JOB AD TECHNOLOGIES (AS OF JUNE 2011) 


COLLECTION 

DE- 

DUPLICATION 

EXTRACTION 

AND 

ANALYSIS 

JOB 

MATCHING 

NOTES 

Indeed 

http://blog. 

indeed.com 

s! 




Free job board (aggregated) 

Job/skill trends by keyword 

Job-posting change (#) over time (month, 
quarter, year), for the nation, and by 
major industry 

50 metro area analyses of job trends 
and job-to-jobless ratios (number of 
unemployed-to-job openings) 

Simply Hired 5 

n/ 




Free job board (aggregated) 

Job-posting change (%) over time 
(month, year), for nation, by major 
industry, and by major occupation 
categories 

50 metro area analyses of hiring change, 
top hiring companies, job-to-jobless ratios 

Geographic 

Solutions 

www.geographic 

solutions.com/ 

solutions-alma. 

asp 

V 

V 



Fee-for-service job board (aggregated 
and de-duplicated) 

Job-to-jobless ratios 

Customizable reports on industry/ 
occupation distribution, education/work 
experience requirements, location of job 
openings, and employers 
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TABLE 1 CONTINUED 


COLLECTION 

DE- 

DUPLICATION 

EXTRACTION 

AND 

ANALYSIS 

JOB 

MATCHING 

NOTES 

Monster 

http://about- 

monster.com/ 

employment- 

index 

■J 

Also through 
partnership 
with Wanted 
Technologies 

Through 
partnership 
with Wanted 
Technologies 

Through 
partnership 
with Wanted 
Technologies; 

■J 

Also through 
new partnership 
with Trovix 

Free job board (Monster jobs only) 

Job-posting change over time (monthly 
counts, year-over-year percent change) 
for the nation, regions, 28 metro areas, 
and by major industries and occupations 

Supply-demand analysis (comparing 
the volume of job openings to resumes 
submitted to Monster) for major 
industries, as well as education and 
experience levels 

Job-matching services: automated 
resume review and referrals 

Wanted 

Technologies 6 

J 

•J 

•J 


Fee-for-service job board (aggregated 
and de-duplicated) 

Job-to-jobless ratios by occupation in 
major metro areas 

Customizable data on occupations, 
industries, skill demands, and key 
search terms 

Conference 

Board 



•J 


Through a partnership with Wanted 
Technologies, the Conference Board 
conducts economic modeling analyses on 
job ad data to produce hiring trends over 
time through Help Wanted Online 7 

Seasonally adjusted changes (#) in job 
postings over month, for the nation, 
regions, states, and major metro areas, as 
well as major occupation groups 

Burning 
Glass — 
EmployOn 

www.burning- 

glass.com 

J 

•J 

x/ 

n/ 

Fee-for-service job board (aggregated 
and de-duplicated) 

Customizable data on skills, education/ 
certifications, work experience, and 
information on salary and job openings/ 
type; available by geography, time 
period, industry, occupation, or level of 
educational attainment, among other 
indicators 

Job-matching services: automated review 
of resumes (for employers), resume 
building (for jobseekers), and career- 
exploration and skill-gap analyses 
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Step 1: 

Collecting Job Ads 


Just as Kayak.com collects flights and fares from multiple airlines and other online travel services and merges 
them on a single, searchable website, enterprising firms have developed ways to pull job ads from thousands of 
websites, including job boards, company career sites, newspapers, nonprofits, and government agencies. Through 
a technigue called spidering, firms can automatically collect job postings on a regular basis-sometimes even daily 
for active job boards or weekly for other sites. Not all spidering technologies are created egual: some firms are 
better able to locate and collect job openings on websites than others. 

Some firms (e.g., Indeed, Si m ply Hired) make aggregated job postings available on public, searchable websites; 
other firms (e.g., Geographic Solutions) provide aggregated job listings through job banks that they administer for 
state governments. In either case, users can browse millions of current job ads and filter those searches by such 
criteria as location, job title and type, company name, salary, or education and work reguirements. 

Step 2: 

De-duplicating Job Ads 

Employers and recruiting agencies tend to advertise job openings on multiple websites and in multiple formats. In 
addition, some companies specialize in collecting and reposting job ads from other websites. Thus, the same job 
opening can show up on hundreds of places online; if each location were counted independently, it would create 
a vastly inflated picture of job demand. To create a more accurate picture of available jobs, firms like Wanted 
Technologies and Burning Glass have developed ways to eliminate duplicates. 

Some job aggregators can only eliminate duplicate job postings that are word-for-word identical. More advanced 
technologies can track down and delete duplicates that are formatted and written differently. They convert 
key elements within job ads to data and then compare the ads against a database of job-listing information. 

Data elements that are commonly coded include job title, location, employer, date of advertisement, and job 
description. 

Step 3: 

Gathering Intelligence from Job Ads 

The final step is extracting and analyzing labor market information from job ads. The sophistication of data 
extraction and analysis varies greatly. Some firms use simple keyword searches as a blunt tool for categorizing 
job ads by job title, location, and employer, as well as preferred educational degree, skill, or some other special 
interest, such as "green" or "social media." At this level of sophistication, firms typically present general 
recruitment trends based on raw numbers of job postings. 

Other firms build off their de-duplication processes to more accurately search and extract key data elements. 

They categorize jobs by a specified occupation code (Standard Occupational Classification-SOC) and industry 
code (North American Industry Classification System-NAICS). This makes job ads a tool for providing real-time 
demand for a particular occupation in a specified area (see Figure 1 on page 8). 
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FIGURE 1. 

HIRING ACTIVITY BY SPECIFIC OCCUPATION AND AREA, MAINE 
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Source: Maine Department of Labor; analysis conducted by Wanted Technologies 


FIGURE 2. 

HELP WANTED ONLINE INDEX OF EMPLOYMENT AND LABOR DEMAND, MAY 2005-MAY 2011 
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Source: The Conference Board, U.S. Bureau of Labor Statistics 
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Firms also run job data against economic models in order to seasonally adjust hiring figures. For instance, The 
Conference Board, in partnership with Wanted Technologies, issues a monthly report, Help Wanted Online, that 
provides a longitudinal time series of hiring activity dating back to May 2005 ( Figure 2 on page 8). 

In addition, firms typically combine job data with government and proprietary information sources for more 
meaningful labor market analyses, such as the supply and demand for skilled labor. Monster has teamed up 
with Wanted Technologies to compare job ads against its massive resume bank to assess workforce shortages 
and surpluses within local labor markets ( Figure 3). For example, a matrix developed for the WIRED Initiative in 
Connecticut and New York show a high volume of IT job openings, coupled with a low supply of job applicants 
with IT skills. This would suggest a market for training IT workers. On the other hand, there was a high demand in 
business services occupations but also a high supply of candidates. 


FIGURE 3. 

SUPPLY-DEMAND MATRIX FOR CONNECTICUT-NEW YORK WIRED INITIATIVE, 
JANUARY-DECEMBER 2008 
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Source: Blue-Green Research Institute: analysis by Monster 
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On the cutting edge, firms like Burning Glass and Trovix use "artificial intelligence" technologies to capture and 
infer greater detail from job postings. Al technologies are based on mining, interpreting, and cataloging the 
meaning of text within job ads-a process that becomes "smarter" with every job ad reviewed. Through a process 
called text segmentation, Al tools can distinguish between sections of a job ad that likely contain qualifications 
information versus compensation. Thus, an employee benefit of child care is less likely to be wrongly construed 
as a desired skill or work requirement. 

Compared with keyword searches, Al makes it possible to accurately extract and code many more facts from 
each job ad, including work experience, desired educational degrees, general and occupation-specific skills, 
required certifications and licenses, and salary and benefits. Al tools also can infer education requirements for 
jobs when none are listed: they match job ads with others that have listed similar skill requirements or education 
preferences. 

Firms using artificial intelligence also are better able to de-duplicate, code, and categorize job ads written in free- 
text format, like those posted on Craigslist. Al tools standardize terms within job ads, like job title and employer, 
making it easier to identify pesky duplicates. The technologies also are better able to recognize when multiple job 
openings are advertised within one advertisement. 


Using Job Ad Data to Support Skill Matching 

In addition, firms are leveraging data extraction and analysis techniques to help match qualified jobseekers with 
employers. Burning Glass and Monster (through its recent acquisition of Trovix) use Al tools to compare skills 
listed in job ads against those described in resumes uploaded to job boards. This makes it possible to identify 
jobseekers who fit an employment need. Through the automated process, employers and jobseekers get notice 
of a potential match. Burning Glass also has tools for helping jobseekers describe their skills and experience 
effectively in a resume and to identify skill gaps in their desired career paths. 
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EXAMPLES FROM THE FIELD 


■ ~ he use of online job ad analysis is growing. An increasing number of state governments, colleges, and 

j other organizations have purchased licenses to analytic tools and reports in order to enhance their 
understanding of local labor markets and support programmatic decisions and investments. California, 
Delaware, Florida, Illinois, Michigan, Oregon, Pennsylvania, Texas, and Washington are among the states 
that have accessed Helped Wanted Online data (Holm, Bergman, & Prince 2010; Altstadt 2010). Ohio and 
B Indiana are working with Wanted Technologies, while Minnesota and Arizona have brought in Burning 
Glass (Altstadt 2010). Florida, Pennsylvania, and the Mid-Atlantic Regional Collaborative-a consortium of 
Maryland, Virginia, and Washington, DC-have purchased licenses to Geographic Solutions to support state job 
banks and real-time analyses. The collaborative is developing a publicly accessible portal for the region's "green" 
job openings, as well as related education/training programs, career services, and labor market analyses. 

In several states, local Workforce Investment Boards, economic development agencies, and education institutions 
have purchased customized labor market reports from Monster (Stephen 2010). In Indiana, Monster data have 
been used in green job studies and to identify top skills and certificates within the traditional energy sector- 
informing the development of energy-specific courses by community colleges. A local WIB in Connecticut has 
incorporated Monster data into successful grant applications for workforce development programs. 

Ohio has plans to use Trovix artificial intelligence technology to extract and analyze the gualifications of 
jobseekers posting resumes on its Monster-supported state job bank. Economic development officials will use 
the skills data to attract employers to the state. Ohio may eventually use the Al technology to analyze job ads. 

Five examples from the field stand out: 

The Maine Department of Labor; 

The Florida Agency for Workforce Innovation; 

> The Northeast Consortium; 

The New Jersey Council of County Colleges Consortium for Workforce and Economic Development; and 
Jobs for the Future's Counseling to Careers initiative. 

The Maine Department of L was an early adopter of tools for collecting and using real-time labor market 
information. In 2008-09 the state partnered with the Conference Board, Wanted Technologies, and Burning Glass 
to improve labor market analyses in a variety of ways. Through the use of the technologies, Maine compares 
monthly changes in online job posts to the numbers of workers employed. In addition, the state can analyze 
preferred gualifications from job ads to determine the number of openings requiring a specified education 
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and experience, as well as certifications and skills in demand. According to analyses for the first half of 2010, a 
Bachelor's degree was the leading educational reguirement ( Figure 4). Finally, the technologies make it possible 
for Maine to categorize job openings by occupation, which, as a low-population state, it cannot do through 
federally sponsored statistical programs. 


FIGURE 4. 

EDUCATION AND EXPERIENCE REQUIREMENTS FOR JOB OPENINGS, MAINE, JANUARY-JUNE 2010 





Associate's Degree 


Graduate Or Professional Degree 


High School/Post Secondary 
Education 


Bachelor's Degree 



o% 5% io% 15% 20% 25% 30% 

. <-1 ■ 1 to 6 Years ■ 6+ Years 

Experience Scale 


35% 40% 45% 50% 

Less than 1 Year 


Source: Maine Department of Labor; analysis by Burning Glass 

Through a simple comparison of job ads to records kept for unemployment insurance recipients, Maine can 
uncover major skill mismatches. Utilizing Burning Glass, the state found a major shortage of workers qualified 
to take substantial number of job openings in the health care sector, while there were significant surpluses 
of workers with experience in several other occupational clusters, notably office and administrative support, 
construction, and production ( Figure 5 on page 14). 
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FIGURE 5. 

COMPARISON OF UI CLAIMANTS AND JOB POSTINGS BY MAJOR OCCUPATIONAL GROUR MAINE 



Source: Maine Department of Labor: analysis by Burning Glass 

Analysis by Wanted Technologies reinforced these findings, while providing occupation-specific employment 
figures. Health care accounted for nearly one-quarter of all job openings, surpassing all other occupational fields 
(. Figure 6; see also Figure 1 on page 10). 


FIGURE 6. 

BREAKDOWN OF JOB OPENINGS, BY OCCUPATIONAL CLUSTER, MAINE 



Source: Maine Department of Labor: analysis by Wanted Technologies 
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The Florida Agency for Workforce Innovation assesses labor shortages and surpluses, utilizing both 
providers of real-time labor market information and government-produced data. The state shares guarterly 
findings with local WIBs to support their reemployment and training efforts. To determine current and ongoing 
demand for workers in particular occupations, Florida compiles data from Help Wanted Online and Geographic 
Solutions, combining that with long-term occupational projections and other government statistical data. The 
demand-side data are compared against a range of sources representing workforce availability, including One- 
Stop Career Center enrollees, secondary and postsecondary students and graduates, and individuals registered 
on the state job bank. The state compares the skills, experience, and gualifications that can be gleaned from 
workforce records to the jobs data to determine whether occupations with job openings have an under- or 
overabundance of gualified jobseekers. Local WIBs use this intelligence to adjust their lists of education and 
training programs that qualify for subsidies. In addition, the state and local areas report hiring trends to the 
media and in monthly press releases. 10 

As part of the State Labor Market Information Improvement Grant Program, the U.S. Department of Labor 
awarded just under $4 million to a consortium of Northeastern states and other partners to identify real- 
time demand for emerging green jobs and skills. The Northeast Consortium includes Connecticut, Maine, 
Massachusetts, New Hampshire, New Jersey, New York, Rhode Island, and Vermont, as well as the Conference 
Board, the Georgetown University Center on Education and the Workforce, the Direct Employer Association, the 
National Association of State Workforce Agencies, and the National Labor Exchange partnership. The consortium 
has developed a definition for green jobs and is working with Burning Glass to aggregate, de-duplicate, and 
analyze online job postings to determine the skill reguirements, occupational demand, and "greenness" of 
available jobs (Georgetown University Center on Education and the Workforce 2010). 

According to preliminary results tallied in 2010, green jobs have accounted for a surprisingly low 2 to 3 percent 
of all openings posted online in the Northeast. But despite the paltry job numbers, the consortium has identified 
significant trends in occupational and skill requirements: a Burning Glass analysis uncovered a larger inventory of 
green-related skills than identified through previous research by 0*NET (Dierdorff 2009). This was accomplished 
by filtering job ads against 0*NET-identified skills and analyzing those ads for additional skills related to green 
jobs. Burning Glass then refiltered all job ads against the larger skill inventory to find many more green jobs. 

In fact, two-thirds of all green jobs identified by Burning Glass would not have surfaced through the 0*NET 
research. This included job openings for technical service managers, sales representatives, and interior designers 
that reguired green skills. 

Burning Glass analysis has categorized green jobs by eight occupational clusters, identified prominent green-skill 
requirements and certifications, and determined the distribution of green jobs by educational level, their relative 
demand for work experience, and wages offered. For instance, the analysis found that nearly half of all green 
building and installation job openings require less than a four-year degree ( Figure 7 on page 16). 
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FIGURE 7. 

EDUCATIONAL REQUIREMENTS BY OCCUPATIONAL CLUSTERS IN THE U.S. NORTHEAST, 
JANUARY-JUNE 2010 


Renewable/Alternative Energy 
Green Research & Development 
Green Promotion & Advocacy 
Green Processes 
Green Building & Installation 
Environmental Regulation & Remediation 
Energy Efficiency Analysis 
Conservation & Natural Resource Management 



■ High School Diploma ■ Post-Secondary/Some College Bachelor's Degree ■ Graduate/Professional Degree 


Source: Burning Glass 

Additional analysis has identified key occupations and skills for workers with education below a Bachelor's degree 
(sub-BA, for short). According to the research, environmental science and protection technicians lead all other 
occupations in offering verifiable green jobs for sub-BA workers ( Figure 8). Financial skills were cited as the most 
common skill requirement in advertisements for energy-efficiency jobs available to sub-BA workers (Figure 9 on 
page 17). Burning Glass differentiated jobs in occupational clusters by a hierarchy in desired qualifications: for 
instance, the analysis segregated an energy engineer from an entry-level energy auditor based on differences in 
skills, academic degrees, and qualifications required. 


FIGURE 8. 

TOP GREENING OCCUPATIONS FOR HIGH SCHOOL AND SUB-BA JOBSEEKERS IN THE U.S. 
NORTHEAST, JANUARY-JUNE 2010 


Environmental Science and Protection Technicians 
Training and Development Specialists 
Industrial Engineers 
Shipping, Receiving, and Traffic Clerks 
Medical and Clinical Laboratory Technicians 
Computer Hardware Engineers 
Biological Technicians 
Maintenance and Repair Workers, General 
Property, Real Estate, and Community Association Man 
Receptionists and Information Clerks 
Industrial Engineering Technicians 
Production, Planning, and Expediting Clerks 
Truck Drivers, Light or Delivery Services 
First-Line Supervisors/Managers of Food Preparation 
Heating and Air Conditioning Mechanics and Installer 
Education Administrators, Postsecondary 
Mechanical Engineering Technicians 
Electrical and Electronic Equipment Assemblers 
Electronics Engineering Technicians 
Medical and Health Services Managers 



■ Verified Green Job Postings as a Percentage of Total Occupational Postings 


Source: Burning Glass 
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FIGURE 9. 

TOP SPECIALIZED SKILLS FOR HIGH SCHOOL AND SUB-BA JOBSEEKERS IN THE U.S. NORTHEAST, 
JANUARY-JUNE 2010 



Source: Burning Glass 


More recently, the Northeast Consortium has begun to examine labor demand in other industry sectors, including 
information technology. Similar to the green job research, Burning Glass has filtered job ads by established IT 
skills and scanned that subset of job ads for other skill sets. This process has proved successful at uncovering 
emerging skill reguirements in IT, a sector known for constant change. The consortium has categorized IT skills 
into eight clusters. According to the analysis, database warehousing is the most common skill reguirement 
(Figure 10). 


FIGURE 10. 

TOP IT SKILLS IDENTIFIED IN JOB ADS, JANUARY-MARCH 2011 


IT Support 
3 % 


Software Testing 
&QA 
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Web Design & 
Technologies 
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Intelligence 
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Systems 
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Databases & Data 
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& security 
12% 


Source: Burning Glass 
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Among other states receiving federal Labor Market Information Improvement Grants to assess labor market 
demand for green jobs and skills, izon also brought in Burning Glass to analyze online job ads. The state is 
comparing Burning Glass data to a recently completed job vacancy survey to assess the validity of both sources 
and to enhance knowledge about the clean energy economy. 

Community colleges in New Jersey are using online job ads as a first step toward identifying opportunities to 
train jobless workers for difficult-to-fill jobs. This is made possible through a partnership between the 

Jersey Council of County Colleges Consortium for Workforce and Economic Development and the 

New Jersey Department of Labor and Workforce Development, using job ad data from EmployOn, a subsidiary of 
Burning Glass. 13 

New Jersey's labor department produces reports on the top 25 occupations with job openings. The job trends 
are supplemented by government data to present a range of information on skill and education reguirements, 
wages, current and projected employment, and unemployed workers with experience in those occupations. The 
findings are available at the state level every month and by Workforce Investment Board region every six months. 

This real-time labor market intelligence piqued the interest of the state community college consortium. It now 
uses a monthly run of EmployOn data to identify training opportunities for occupations with significant hiring 
in a particular sub-state region, usually a three-county area. Once potential leads have been identified, the 
consortium's job developers contact the employers listing the jobs. The developers validate the job openings and 
determine whether the employers are having a hard time finding qualified workers. This process has eliminated 
from contention some seemingly hot jobs, like IT jobs, which are flooded with out-of-work applicants. If enough 
employers express a need for workers in a particular occupation, the consortium develops a customized training 
program geared toward preparing unemployment insurance (Ul) recipients to fill open jobs. 

The consortium heavily involves employers in verifying the skill requirements, designing the training curriculum, 
and screening potential trainees. The public workforce system is brought in to recruit Ul recipients and determine 
their eligibility for Workforce Investment Act training funds, which pay for the training. One community college 
in the region is selected to deliver the program. So far, about 100 Ul recipients have been trained for jobs as 
electronic technicians, bank tellers, and electromechanical engineering technicians-and in emerging health care 
occupations (e.g., patient service representatives). The placement rate is 82 percent. 

Through Counseling to Careers, Jobs for the Future has been working with community colleges in North 
Carolina to use traditional and real-time labor market data to guide students toward "best-bet” career pathways 
and associated community college programs. 

With the assistance of JFF, community colleges are collaborating with public workforce systems, school districts, 
and other partners to comb through EmployOn and government statistical data and engage employers in search 
of best bet occupations that meet the following criteria: they have significant job openings in their particular 
region: require sub-BA credentials; pay wages at or above the local median; and are projected to grow. They 
also examine the capacity and success of community college programs to prepare students for these best bet 
occupations. Their efforts are expected to result in the development of fact sheets that counselors can use with 
students to help them make more informed decisions. 
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LIMITATIONS ON ANALYZING ONLINE 
JOB ADS 


D espite the increasing use of and interest in analyzing online job ads for real-time labor market 

intelligence, the practice suffers from significant limitations in the data and weaknesses in the analytical 
technologies. As the building blocks for analyzing real-time labor market demand, online job ads have 
three substantial shortcomings: 

> Not all job openings are posted online, distorting the employment picture. 

Deriving an accurate count of job openings posted online, comparable over time, is not yet possible 
because current technology cannot eliminate all duplicates and find all ads. 

Few online job ads include complete information about desired qualifications. 

Holes in Hiring Activity 

On its face, online hiring activity might appear to be more than sufficient to provide meaningful analyses of 
the labor market. However, job ads do not provide an accurate employment picture across all industries and 
occupations. Research suggests that job openings in retail, food service, building maintenance, and construction 
are substantially undercounted online: those employers find workers by other means. Job activity in other 
sectors, such as information technology, health care, and management overall, are captured more completely 
online. According to researchers, the proportion of job openings advertised on the Internet would rise from 70 
to 95 percent when excluding underrepresented construction and service-sector occupations. 


Flaws in the Data and Analysis 

The job openings that are advertised online are done so in many places, but the ability of various technologies to 
catch and eliminate duplicate ads on aggregated sites differs greatly. Estimates vary on the number of duplicates 
that exist in job data. Burning Glass reports that at least half of all job ads that it collects are duplicates and that 
it eliminates them. Technologies that compare the full text of job ads miss at least one-quarter of duplicates. 

In calling up employers and culling through job ads, job developers in New Jersey uncovered significant numbers 
of duplicates-as much as 40 percent of the listings for some occupations. Meanwhile, duplication appears to be 
less of a problem for the Northeast Consortium project. State labor market information agencies participating 
in the project double-checked a small sample of job ads that had already gone through Burning Glass's de- 
duplication process and found that only 3 percent of ads were duplicates (Jayasundera 2011). 
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As a result of flaws and discrepancies in "spidering" and de-duplicating technologies, firms are reporting 
wildly different job counts across the country, raising concerns about the validity of the data and the process. 
Recognizing these deficiencies, real-time providers are improving their processes for collecting and cleaning their 
data. In January 2011, the Conference Board reduced its monthly job totals by about 11 percent after improving 
de-duplicating methodology, eliminating invalid job ads, and adjusting coverage of job boards. Monster has 
decided to hire an independent auditor to review its data and release the results to the public. Researchers for 
the Northeast Consortium uncovered a glitch with Burning Glass's data scrapers, which had caused a sudden 
drop in job listings in late 2009 to early 2010. The problem has since been fixed. Meanwhile, Burning Glass has 
improved its spidering technology through the acguisition of EmployOn, which has tripled the number of websites 
scanned for job openings. 20 As a result, researchers for the Northeast Consortium have reported a significant 
increase in job openings through the Burning Glass data. 

It remains unclear what portion of increasing job ad levels should be attributed to actual hiring activity versus 
improvements in spidering technology. Growth of job ad levels may just reflect that more employers are turning 
to the Internet to post their job openings. In particular, free sites like Craigslist have attracted many jobseekers, 
and technologies like Burning Glass are becoming more adept at capturing those types of ads. 21 

These continued changes in technologies and methodologies call into guestion both the reliability of reports 
on hiring trends and predictions of future employment needs. The continuous improvements to job ad analyses 
hinder reliable comparisons of labor market demand over time. This sort of time-series analysis is common 
among federal statistical programs because of continuity in their data collection methods; in the case of its 
monthly reports on employment, the government uses economic models to adjust jobless rates based on 
seasonal differences in hiring activity. Firms such as the Conference Board also use economic models to adjust 
job ad levels reported through Help Wanted Online in an effort to improve the reliability of comparing listings 
over time. 

Still, researchers suggest that analyses of online job ads may simply provide a signal for employment trends 
three to six months beyond their release. The short-term nature of reliable projections is enough for education 
institutions to develop and deliver short-term training programs to fill immediate workforce needs. However, 
institutions may not have enough time to realign and ramp up degree programs, unless employers indicated an 
ongoing skill need. Furthermore, most job ads are not detailed enough and analytic technologies are not yet 
smart enough to return reliable intelligence on hiring needs within a particular industry subsector or zip code. 
Researchers advise keeping job ad analyses to two-digit NAICS codes and at geographic areas no smaller than 
metropolitan areas. 


Missing Information on Skills 

It is a relatively straightforward process to gather information in job ads that have fields for job title, location, 
employer, education reguirements, and other information. But it has proven challenging to extract detailed 
information from job ads. Those written in free text format, like those published on Craigslist, are difficult to 
analyze. Moreover, many employers do not identify education reguirements or other qualifications. Research 
conducted for the Northeast Consortium found that less than 40 percent of ads contained information on desired 
academic credentials. Some omissions are easily explained: employers seeking a physician or lawyer may not see 
the need to identify a M.D. or J.D. as a prerequisite for employment. 
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As mentioned previously, artificial intelligence technologies are increasingly able to infer education requirements 
in job postings that require similar skills; this cutting-edge analysis can capture perhaps another 10 percent of 
job postings. However, these technologies are not sophisticated enough to infer academic degree, education, 
or certification requirements in the absence of a similar job ad for comparison. For most job openings, the lack 
of preferred academic credentials means data users, including community colleges, cannot make meaningful 
assessments about the qualifications needed to perform particular occupations. 

Additionally, artificial intelligence has limitations on uncovering emerging skills; Al tools only can extract skills 
that are added to search lists. If real-time analysts are unaware of new skill sets-and these are constantly 
emerging in information technology and some other sectors of the economy-then Al tools will not extract them 
from job ads. 

Even when education and skill requirements can be captured or inferred, there is still the question of whether 
online hiring activity represents true labor supply and demand. Research suggests that skill requirements in job 
ads may not accurately represent employer demand; rather, the ads are more likely to require higher skills than 
other available jobs within the same field that are not posted online. This is also because job postings generally 
tend to seek out the best possible-even overqualified-candidates. In particular, this puts into doubt whether most 
green jobs require the high levels of skills reported by Arizona and the Northeast Consortium. Additionally, just 
as job ads skew toward higher-skill positions, jobseekers who post resumes online tend to be more qualified (and 
older) than the U.S. labor force as a whole. Analyses of workforce shortages and surplus need to take this into 
account. 
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PROMISE AND POTENTIAL 


E merging technologies that collect and analyze job openings advertised on the Internet have the potential 
to provide community colleges with rich arrays of real-time intelligence about the occupational and 
skill demands in their local labor markets. These tools have helped guantify demand for green jobs 
and emerging skill requirements while uncovering labor market mismatches, customized training 
opportunities, and best-bet occupations. Yet gaps in available data and constraints in the technology have 
hamstrung the promise of real-time labor market information. Of principal concern, not all job openings 
are posted online, duplicate ads are rampant and difficult to eliminate, and few online job ads include complete 
information about desired qualifications. 

Any and all approaches for assessing labor market demand have their limitations. Data collected from online 
job ads are best used to complement-not replace-other ways that community colleges can identify labor 
market needs. Pairing online job ads with government-backed statistics and projections and business outreach 
efforts can dampen shortcomings inherent with each approach and provide the best insights on immediate and 
ongoing occupational and skill demands. While online job ads are better for uncovering immediate job openings 
and emerging skill requirements, government statistics offer a broader assessment of economic conditions, 
such as employment levels, earnings, and the number of establishments within industry sectors. Moreover, 
cross-referencing skill information from online job ads with 0*NET data produces better intelligence about the 
qualifications needed to perform specific occupations. Additionally, utilizing both real-time and traditional labor 
market information opens up new forms of analysis. Matching job-ad data to Unemployment Insurance records 
can reveal workforce shortages or surpluses for in-demand occupations. 

Although collecting data from online job ads can be an efficient way to gather intelligence on local workforce 
needs, it does not replace direct communication with employers. Only by collaborating with employers can 
community colleges know for certain which occupations and skills are in demand and whether or not there are 
enough qualified workers to fill available jobs. On the other hand, online job data can validate and enhance labor 
market analyses conducted by industry advisory groups and sector partnership initiatives-ensuring that their 
findings represent the needs of local employers who are not sitting around the table. Job ads also can provide 
leads on employers who may be interested in customized training and apprenticeship solutions. 

By taking such a comprehensive approach to analyzing their local labor markets, community colleges can tailor 
programs and course content to their markets and ensure that students are prepared to build marketable skills 
for growing careers. 
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